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Abstract 

Crawling the Web to build collections of documents related to 

pre-specified topics became an active area of research during the 
late 1990's, crawler technoiogy having been developed for use by 
search engines. Now, V^eb crawiing is being seriously considered 
as an important strategy for building iarge scale digital libraries. 
This paper covers some of the crawl technologies that might be 
exploited for collection building. For example, to make such 
collection-building crawls more effective, focused crawling was 
developed. In which the goal was to make a "best-first" crawl of 
the Web. We are using powerful crawler software to implement a 
focused crawl but use tunneling to overcome some of the 
limitations of a pure best-first approach. Tunneling has been 
described by others as not only prioritizing links from pages 
according to the page's relevance score, but also estimating the 
value of each link and prioritizing them as well. We add to this 
mix by devising a tunneling focused crawling strategy which 
evaluates the current craw! direction on the fly to determine 
when to terminate a tunneling activity. Results indicate that a 
combination of focused crawling and tunneling could be an 
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Advanced Internet Searching 
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